Open access to clinical trials data.

Well-conducted randomizedclinical trials (RCTs) are thegold standard for evaluating the safety and efficacy of medical therapeutics. Yet most often, a single group of individuals who conducted the trial are the only ones who have access to the raw data, conduct the analysis, and publish the study results. This limited access does not typically allow others to replicate the trial findings. Given the time and expense required to conduct an RCT, it is often unlikely that others will independently repeat a similar experiment. Thus, the scientific community and the public often accept the results produced and published by the original research team without an opportunity for reanalysis. Increasingly, however, opinions and empirical data are challenging the assumption that the analysis of a clinical trial is straightforward and that analysis by any other group would obtain the same results.1-3 In this issue of JAMA, Ebrahim et al4 report their findings based on a rigorous search of previously published reanalyses of RCTs. Their first surprising and discomforting finding was just how infrequently data reanalysis has occurred in medical research. Searching the literature from 1966 to present, the authors found only 37 reports that met their criteria as an RCT reanalysis. Of these few reanalyses performed, the majority (84%) had overlapping authors from the original report. Thus, reanalyses are not only rare, but the majority that were reported were not fully independent of the original research group. Despite this overlap, Ebrahim et al report that about half of the reanalyses differed in statistical or analytic approaches, a third differed in the definitions or measurements of outcomes, and most important, a third led to interpretations and conclusions different than those in the original article. While the definition of what constituted different trial analyses, study end points, findings, and interpretations is subjective, the authors’ general conclusions were consistent with an emerging literature that indicates RCT reanalysis can yield different results and conclusions from those originally published. Even when the original investigators are presenting evidence in different venues it is not always consistent. For example, there is evidence from trials that data presented to the US Food and Drug Administration (FDA)may differ in important ways from those originally presented at scientific sessions or published in medical journals. Rising et al5 assessed clinical trial information provided to the FDA and reported a 9% discordance between the conclusions in the report to the FDA and in the published article. Not unexpectedly, all were in the direction favoring the drug. Another example is discordance between what is reported in ClinicalTrials.gov and what is published in journal articles.Hartungetal2 showedthat inarandomsampleofphase 3 and 4 trials, in 15% the primary end point in themain article was different from the primary end point the trialists reported in ClinicalTrials.gov. Moreover, 22% reported the primaryoutcomevalue inconsistently,withsomeevenhavingdifferences in the number of deaths. Other studies have found similar rates of discordance.6 When reanalyses by different groups obtain somewhat different results or reach alternative conclusions, the cause is not necessarily bias. Independent groups with individual patient-level data from trials will not always reach the same conclusions because every study involves discretionary decisions. This situation was exemplified in the evaluation of Medtronic’s bone morphogenetic protein 2 (BMP-2) product. Two expert organizations with international reputations in the conduct of systematic reviews were given the same individual patient-level data from the BMP-2 trials, the same task, the same resources, and the same timeline.7,8 Nevertheless, in their final reports, their methods were not identical and their results and conclusions differed in important ways. Consistent with these findings, meta-analyses on similar topics and using similar data do not always show concordant results.9 The current reviewbyEbrahimet al, aswell as other cited work, suggests several important next steps needed to ensure transparencyandopenaccess inRCTs.First, allRCTs, their prespecified study protocols and analytic plans, and their results should be registered and reported to the medical community, fulfilling the ethical promise made to those enrolled in the scientific experiment. Such a step would contribute to improvements in the standardization of trial registration and reportingof results,which remainsvarieddespitegovernmental regulation and journal policies.10-12 Full availability of trial registration data is essential to allowpeer reviewers and journals tomonitor trial protocols andanalyticplans toensureconsistencyand thereby reduce someof thevariation thatmayoccur in the reporting of results, particularly with respect to primary, secondary, and exploratory outcomes. Second, rawdata andmetadata (all the information about the data) from the original trial should ideally bemade available to those who seek the opportunity to replicate the findings. Such independent verification would markedly increase the scientific community’s confidence in the study findings. Evenwhen results differed importantly, it would allow for open dialogue that would promote a deeper understanding of the study and its interpretation. While the current study by Ebrahim et al demonstrates how infrequently Related article page 1024 Opinion